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Assessing the Quantitative Skills of College Juniors 

Steven F. Bauman 
William O. Martin' 



Two Quantitative Tasks 

Suppose we gave the following problem to a group ol juniors at your local college or university — what proportion 
of the students would you expect to give the correct answer? 

Problem L A media professor asked the students in his class whether or not they read Time or 
Newsweek the previ(>us week. The students' responses are summarized in this table: 

Time 



Yes 

Newsweek 

No 



How many students said they read Time 

We have used this and similar problems with iiiain students at the University of Wisconsin-Madison and were 
surprised to find that only about a thirti of the suidents. ami rarely more than lialf, correctly answered this question 
about data in a cross tabulation table. 

Ixt’s consider another, more mathematical, problem. Suppose this time that a group of juniors, who had studied 
from one to three semesters of calculus, were given iVohlcm 2. Hovs ilo you think they would do/ 



Yes No 



4! 


23 


52 


10 



the previous week'/ 




' Steve hauman is a lYotessoi ijf Mathematics at the UniversitN ot W’lscoiism-Madison and has directed the 
Oiiantitativc Assessment lYoject since 1^)91. Mill Martin completed his Ph. D. at UW-Madison and is presently a 
faculty member at Ninth Dakota State Uimersit\, He worked on the quantitative assessment project from its 
inception in l^^O, first as a graduate student m mathematics and mathematics education, and later as a researcher 
I'his paper is an expanded version of the article puhlished in the MAA's C'ollej^e Mathematics Journal in May l^d5 
(volume 26 number } pages 214-220), 
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Figure 1. 



Problem 2. Figure I is the graph of a fioiaion y ■■ /(,v), I 'sc the graph to estimate these values as 
accurately as you ran: 

a. /(3) h. f (l) c. /(I) 

(L For which values (f x in the interval (O.^j is f (.v) negative'/ 

The proportion of students in our study who correctly answered a group of questions about a graph, such as this, 
ranged from 10% (in a course where most students had completed just one semester of calculus) to around 60% (in 
a course for students who had completed the standard three-semester science and engineering calculo*' sequence). 
Again, we were very surprised that so many college juniors had difficulty with such questions. 

Our research addresses the extent to which emerging college juniors have the quantitative skills required for 
success in their chosen upper-division courses. This article describes our quantitative assessment project, its findings, 
and its impact. Our assessment project may be of special interest to mathematicians because it has been implemented 
by mathematicians and statisticians, iiol assessment professionals. 'I'he process has several important advantages over 
more familiar, standardized externa) assessments, (a) It is faculty driven, promoting faculty involvement and 
rcHecting faculty needs (see 12] and |4]), (b) the assessment process and its outcomes contribute equally to 
instructional improvement; (c) it has a specific focus and is tied to particular courses; (d) it reflects the educational 
choices and needs of students; and (e) it provides useful information both for participants and for departments and 
campus adm.inistrators. 

The tw'o problems given above may already have raised some questions (or doubts). For instance: 

► Are the reported results representative of college juniors generally? Although our approach is closely tied to 
specific courses, repeated assessments reveal similar [)atterns in a variety of settings. We arc confident that our 
findings are at least representative of undergraduates at this university, and most likely they apply more widely. 

► Why use these problems — what makes these quantitative skills important? All test problems arc chosen by 
upper-division undergraduate course instructors as representing skills needed for success in their course, The 
Items may not always reflect wha: mathematicians or statisticians believe is important, but they definitely reflect 
expectations of faculty niemf>ers in otlier departments, 

► Is this ampther discouraging report tpf the inadequacies of American college students? If so, do y<m have 
specific suggestions to help faculty members deal with the problems? Yes. we have found some disc ouraging 
information about the capabilities of undergraduates, and tlie project has identified areas of the undergraduate 
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curriculum that sltould be improved But we believe that individual faculty members and departments will 
respond effectively to this appraisal of their students’ capabilities. Precisely because it is so closely tied to the 
courses they offer, the information we provide should be useful as they plan their responses to problem areas we 
uncover, A side benefit is that our assessment process encourages faculty members to reflect on and discuss with 
colleagues the goals of their instruction. 

Two Roles for the Math Department 

Mathematics has important roles in the undergraduate curriculum that provide the context for our assessment work. 
On most college and university campuses the subject is considered, to some degree or another, an essential 
component of the general education of every student; only English has a comparably ubiquitous role in the 
undergraduate curriculum. Because of this, mathematics faculty members have dual instructional roles in their 
involvement with introductory undergraduate mathematics. On the one hand, the first two years of the curriculum 
provide the groundwork for the upper-division and graduate work of those who will specialize in mathematics. 
Mathematicians, like scholars in other disciplines, are best qualified to decide what preparation their students need 
to pursue advanced study in mathematics. On the other hand, mathematics also functions in a service role, which 
often requires much of the department's instructional resources, Fypical of this imbalance, our department recently 
had about 250 declared undergraduate mathematics majors, while more than 6500 students were enrolled in cours ‘s 
up to introductory linear algebra. A significant problem mathematicians face in undergraduate programs is to design 
instruction that fulfills both instructional roles— preparing majors and serving the quantitative needs of other 
departments. 

This balancing act is a challenge because the roles are not necessarily compatible; tradeoffs must be made. For 
example, .should mathematics departments use specialized, parallel courses, requiring students to make an early 
(perhaps premature) decision about the direction of their academic career? Or should general intrcxiuclory courses 
be used, preserving student options and, perhaps, conserving limited instructional resources — but at the expense of 
the level of preparation for specific disciplines? 

Our Assessment Project 

Like many institutions ot higher education in this era (iccoimtuhility (sec 11)), those in the University of Wisconsin 
system were directed to implement an assessment plan targeting the quantitative and verbal capabilities of emerging 
juniors. The faculty committee that worked to meet this Regents' directive on the Madison campus decided to break 
from existing quantitative and verbal assessments that measure all students on a common scale (e.g., commercial 
standardized tests). Considering the tremendous range in the quantitative backgrounds of college juniors (from no 
collegiate mathematics to advanced undergraduate course w'ork), the group instead sought ways to tie assessment to 
student backgrounds, 'fliis was accomplished by linking assessment to specific courses. 

Perhaps it was not too surprising, considering prevalent methods of classroom assessment, that the oroup selected 
classroom tests to generate the required information. We developed an “itembank" of problems covering material 
from undergraduate niathernatics courses (througli introductory differential equations) and from basic statistics, 
Instructors, selected from a range of departments, use a questionnaire linked to the itembank to identify the specific 
quantitative skills their students need. The students are then given a test at the start of the semester designed to 
determine whether they have these skills. We discoveied that the careful design of these assessment tests is crucial 
and have ta. hired our original plans for an “automated'* test-generating procedure accordingly. 

About M) custom-designed fiee-response (i.e., not multiple choice) tests have been given to ,^(K) or more students, 
mostly juniors, each semester since Fall Tiie tests. usualK gi\en during the second week of classes, assess 

the extent to which students possess those quantitative skids that their instiuctors (a) identify as essential for survival 
111 the course, (b) expect students to iia\e Irom the first day ot cUiss. and (c) will not cover during ttie course. For 
example, both problems 1 and 2 above met these criteria in several courses. Our role as assessors is to ensure that 
each test rellects w’hal stiulents in a course need to know. I'he le^ts are intended to be neither “wish lists ' nor 
comprehensive examinations of the content of prerequisite mathematics courses. 

C'orrected test pajKTs are returned to students, along with solutions and specific relerences for remediation, within 
one week, Instructors receive intormation about the students' test performances a few days later, rhus, early m the 
semester both students and Instructors possess useful information about instructor expectations, student capabilities. 
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and the need for any corrective action. We have developed a reliable grading system that allows mathematics 
graduate students, with limited training, quickly to record information about the students’ work and their degree of 
success on each problem. The coding system provides detailed data for later analysis while allowing the quick return 
of corrected papers to the students. 

Information of two kinds is generated by our assessment process: (a) a detailed picture of those quantitative skills 
needed for upper-division course work in other departments and (b) an assessment of the quantitative capabilities of 
emerging juniors outside the context of specific mathematics courses. The first comes from our personal contacts 
with faculty members as we design the test and interpret the results; the second is provided by analysis of students’ 
performance on the assessment project test md tlieir quantitative backgrounds as shown by university records. 

Departmental Needs and Student Capabilities Reveo^ed by Assessment 

More than 3000 undergraduates have taken an assessment project test over the first five years of operation. Broadly, 
we find that instructors do expect their students to have certain skills covered in prerequisite mathematics or statistics 
courses and that many, even most, students have adequate skills to meet the quantitative demands of their chosen 
upper-division courses. While the project has not revealed gross mismatches at this institution between instructor 
expectations, the content of prerequisite courses, or student capabilities, specific areas require some attention. We 
should caution that the natural inclination to focus on shortcomings, as we did at the start of this article, may convey 
an exaggerated impression of student unpreparedness. 

University transcripts provide supplementary intoimation related to student performance on assessment tests. We 
discovered a group of students who apparently avoid any college courses with a quantitative component judging by 
their transcript information, as well as written student and instructor comments on tests and follow-up questionnaires. 
This behavior may not be inherently undesirable. Bui taken along with the poor performance on routine, basic 
statistics and arithmetic tasks needed in their choser, nonquantitative courses, this finding raises two important 
questions: Is there a base level of quantitative literacy that should be required of all baccalaureate students? If so. 
how should these requirements be set and met? A university curriculum committee pursued these questions, 
ultimately recommending a university- wide quantitative degree requirement. This is probably the clearest example 
of our assessment work's broad curricular impact. 

Three levels of quantitative expectations. There seem to be three le\ cls i)f quantitative expectations for students in 
upper-division undergraduate courses. Level I courses, such as l^hm iples of Advertising and Constmetion of 
Classroom Tests, lack any formal quantitative prerequisites. Their instructors expect basic statistical and arithmetic 
skills along with the ability to read and interpret information presented in tables and graphs. Sometimes Level 1 
courses also draw on basic geomelric and algebraic capabilities. All these quantitative skills are found in high school 
curricula. Whether students have these skills is of amcern tc^ mathematicians, but they have no place in the college 
mathematics curriculum. 

Student capabilities in c(uirses at the other two cjuantitati\ e le\els depend much more on introductory college 
mathematics and statistics courses. Le\el 2 courses, such as Tinanee and Quantitative Methods in Agricultural 
Eamomics, require a semester of calculus ami perhaps a first course in staostics. Level courses usually require 
three semesters of calculus. H.xamples of 1 ,e\el courses that we lui\e assessed include Hiophysical Chemistry. 
Circuit AfUilysis, Technicpics in Ordinary Dilfoential Lquations. ami Mechanics. 

Patterns of student performance on quantitative tasks. Oui results in several i.cvel 1 courses revealed gaps in the 
curricula of the mathematics ami statistics departments. I liat is. students wlio had dilficulties with problems on the 
assessment tests could not be directed to existing courses that cover the material. I'or example, an early and 
surprising iliscovery was that so many students could not sidve problem 1 above, rids skill frt>m basic statistics is 
important to understaml experimental results repiMteil in a journal article. Students in Level 1 courses are usually 
successful with direct computatums le.g.. converting temperatures Irom b'ahreiiheit to C’elsius using a prc^videtl 
tormula) and one-step problems (e g., using the table in problem 1 to say how many students did not read cither 
maga/ine). Many students at this level run into ilitliciiltv when askeil to relate informatum logically, as m problem 
1, or to extract informaiion <itul ilevise a strategv to use it lh(^ ilem ^ is another example of such a task, involving 
percentages. 
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Problem An advertising company is planning the layout for 
a full page ad in a magazine with pages. 

The cost of the ad depends partly on the amount 
of printed space ie.xcluding margins), so they want 
to compare the printed area if they use no 
margins to the printed area using /" margins. 
What percentage of the full page will he printed 
if they use one-inch tnargins all around as shown 
in figure 2 '! 




Figure 2. 



In one Level 1 course. 2iY7( of the students answered this question correctly; in another about one-third got it. 
'I'he difficulty is less in computing percentages than in deciding what to do with the information that js provided. 
When the same classes were told "170 is of a number — find the number/' three-quarters of the students gave 
the correct answer. 

Most students in Level 2 courses are successful with tasks from statistics and precalculus; few' show proficiency 
with materia! from calculus. C'oncepiual tasks, such as deducing information about a function's derivatives based 
on a graph of the function (problem 2). are iu>i handled well. In most Level 2 courses many students can handle 
routine tasks, such as finding a derivative, but have little success with other material from calculus. 

Students in Level 3 courses also have difficulty with similar, less routine, more conceptual problems, At this 
level, a common question involves the evaluation of a definite integral. In one Level 3 course many more students 
could exactly evaluate a definite trigonometric integral symbolically than could accurately estimate its value from 
a graph. In other courses many students were unable to make numerical or graphical approximations of integrals. 

I c our surprise, only a quarter of the students in a first diftereniial equations course correctly evaluated a convergent 
geometric series. Generally. Level 3 students are prepareil for the quantitative requirements of their chosen courses. 
Although students may base difficulties with specific problems, instructors from these technical courses report that 
students usually regain the necessary skills during the semester. 

General Conclusions 

It seems that instructors often want students to be ;ihlc to reason ituiependently, to make interpretations and to draw' 
on basic quantitative concepts in their courses; they are less concerned about whether students remember particular 
algorithms or procedures. I hese conceptual expectations were confirmed during our meetings with groups of faculty 
members from other department'', In contrast, students are most successful w ith routine, standard computational tasks 
and often show less ability ii> use conceptual knowledge or insight to solve less standard quantitative problems. Put 
another way, in tlie context of our exams many students can do what they have been shown, successfully handling 
certain kinds of conventional pioblems (e.g,. using substitution oi miegiatiiMi by parts); few students seem able to 
make connections or to solve more novel problems (e.g., estimating an integral's value from a graph or tabular data). 
Put another way. many students can do what thev have been shown, successfully handling certain kinds oi 
conventional problems (using subsuiution or integration by parts); lew students seem able to make connections or 
to solve more novel problems mi their own m the context olOur exams (estimating an integral's value from a graph 
or tabular data). 
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Another example may illustrate both the general nature of quantitative skills that faculty in other departments often 
want and the way in which our process, through ongoing discussions with individual faculty members from across 
the campus, helps to develop very specific and, we believe, accurate information about the needs of other 
departments. In 1991 an engineering professor asked us to include a Taylor Series problem on his test. As we 
reviewed the results of the first semester’s tests with him, he observed that he had not really made use of Taylor 
Series during the semester. Instead, he suggested another problem for the second semester that better represented 
the quantitative capabilities that students needed for his course. Problem 4 , proposed by this engineering professor, 
illustrates a broader pattern that we have found over the years: Faculty more often want their students to have what 
could be called quantitative or number sense than particular, specialized manipulative skills. 



Hrohlem 4. Use a Inon-^raphiny,, scientific I calculator to help sketch a j^raph of the function 

f{t) - 4e ^ sin r over the interval i) < t < 27 t . Choose scales so that the 
graph makes use of all of the provided grid. 

I A 3-inch square grid for graphing was provided with this problem. I 

Use your graph and calculator to estunate the tninimum value of f(t) over this interval. 

The engineering instructor wanted students to use their own .selection of output values and careful reasoning to 
understand the behavior of a function over an interval —he was more interested in their problem-solving skills than 
in their ability to recall and apply algorithms. This problem also illustrates an important attribute of our approach: 
The tests reflect the instructor's quantitative expectations, not what we as assessors (and teachers of mathematics) 
think students ought to be able to do. No students in the engineering course where this item was used had access 
to graphing calculators — only a quarter of the students were successful with this problem. In comparison, about 15% 
of this same engineering class successfully evaluated j ~te ' dt . 

Another engineering professor, commenting on his students' performance on the a.s.sessment test, observed that 
he was not surprised that his students did well on our test since it cemtained fairly standard mathematical problems; 
he felt that students could “turn the crank," but were unable to connect the mathematics to the physical situation 
being modeled, such as to use an in :ral to model hytirostatic force. 

Impact of Assessment 

Many of the specific results we have obtained are quite interesting in themselves, but an important question is 
whether this work can have any impact on undergraduate curricula and instruction. The quick answer is “yes," and 
in a variety of contexts. However, the individualized approach we use has not produced, and is not likely to provide, 
quick, broadly general izable results that lead to wide scale, dramatic curricular changes. Instead, the ongoing process 
promotes faculty reflection on the goals and impact of their instruction; changes have been, and arc likely to remain, 
incremental and gradual. 

Impact on participants. The clearest potential for impact is m the particular course being assessed. We have seen 
a variety of responses from faculty and students, ranging from no action and indifference through extra review and 
restructuring of courses. It seems we have had the least impact on students. In some technical cour.ses many students 
report studying both before and after the assessment test, but the most common responses are (a) “I already knew 
this, so why waste my time" and (b) “If 1 wanted to be a math major I would have taken a math course — leave me 
alone." The latter is often heard from students who do not appear to have tlie tested skills. Most students, when 
questioned at the end of the semester, recognized that the skills were important in their course, but had not chosen 
t(' use to use tlie assessment information to liclf) with picparaiion for those requirements. 

I Instructors, on the other hand, have mostly reacted very favorabls to the assessment prcKess. Ihose who do not 
leport making any changes eitliei found from tlie tests that students had the prerequisite skills or said that they were 
already aware of the difficulties and had modified then appnnich to deal with them — the project simply confirmed 
what they had suspected. In cases where instructor expectations diflered from the results they often reported making 
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changes, either omitting reviews that no longer appeared necessary or including additional work to develop important, 
missing capabilities. 

Our experiences show that the design of the assessment prtKedure has successfully met the goal of being useful 
and meaningful for participants, although we would like to find ways to increase student gain from the experience. 
What about impact at the departmental and university level? To what extent can a highly localized approach provide 
information at a broader level? It appears that we are having success here. too. 

Departmental impact oj assessment. The potential value to the Department of Mathematics of the data generated 
by assessment is quite clear. We report annually to the entire faculty, but we have probably had greater curricular 
influence by targeting our findings at groups responsible for particular levels or groups of courses, particularly 
precalculus and calculus. Findings from many assessed courses have sbo’vn, for instance, that faculty want students 
to interpret graphical representations. This had not been high';, emphasized in mathematics courses. It was 
somewhat ironic, but instructive, that in an early meeting discuss our findings with a curriculum group in 
mathematics one faculty member remarked about a probh':o like the second one given in this paper, “Fm not 
surprised students couldn't do that — I never ask such quest* ns in my class." A colleague responded that he thought 
such tasks were very important and always emphasi/eu such ideas when he taught calculus. Obviously our 
assessment work, as in this instance, can stimulate valuable discussions about what is and should be covered in 
introductory mathematics courses. Our findings about graphical representations have led coordinators at this level 
to encourage instructors to give increased attention to graphical representations of functions. Perhaps more important, 
though, is what our work shows about the kind of mathematical skills that are needed in other departments: 
Instructors seem less concerned about computational, algorithmic knowledge than more conceptual, problem-solving 
capabilities. This has implications for the way that rnathemc ’cs is taught, the expectations for what students will 
do. and not just the content of matliematics courses. 

Assessment has also influenced participating departments and we have moved recently to improve communication 
at this level. W'hen assessment reveals problems, an effective approach has been to send a summary report to faculty 
members in the affected department. Following the written report, we attend a regular faculty meeting to answer 
questions and discuss the issues raised by assessment. The information we provide could lead to a variety of 
departmental changes (so not all problems require resolution by the Department of Mathematics). In one department, 
a faculty member remarked that students claimed they did not realize they would be expected to know material from 
a prerequisite calculus course in their later course work! A natural response to this situation involves advising: 
F*aculty in the department should ensure that prospecti\e majors understand that prerequisite courses cover important 
knowledge that will be used later; that prerequisites are not just some sort of hurdle placed in a student's path. 

After finding that many students were unable to handle material from calculus, another department beefed up the 
prerequisite course from first semester business calculus to iwn semesters of the regular calculus sequence They 
did this not because the students needed the additional content, hut to ensure that their students had further developed 
the necessary fundamental ideas by using and lev iewing hem in later mathematical work. 

Campus^wide impact oJ assessment, The sinking cjuanti ative deficiencies of students who seemed to be actively 
avoiding any courses with quantitative expectations was mentioned earlier. These findings from some I.evcl 1 
courses contributed to a university curriculum committ'.e recommendation that all baccalaureate degree programs 
include a six-credit quantitative requirement. The recommendation was adopted by tfie F^'aculty Senate, indicating 
that our localized procedure can produce information useful at the broadest institutional levels. 

How do faculty respond when significant numbers of students do not have necessary skills, quantitative or 
otherwise',^ In some cases, w e have found, w ith resignaiion; “It would be lovely if we required three oi even two 
semesters of calculus. Hut one w ill have to do." w as the response from one instructor. Another faculty member said 
he had cho.sen to completeU leave all ipiantiiative material out of his nontechnical course because (a) students lacked 
the necessary skills and (b) he had plenty of other material to cover, In other cases, faculty carry on regardless, 
“Ihe students pick up [the necessary skills] as we go. drop, or perish. ’ It seems that another important function for 
our grtmp is to bring the range faculty responses u* peiceived weaknesses in students' backgrounds to the attention 
o\ all faculty members, imhv a uals, especiallv junior untenured faculty, may feel pressures that lead them to make 
choices that are not in the best .iterest^ of the instil utum. An e.xample we have encountered more than once is 
watering down or eliminating important )uantit uive material in response to perceived student capabilities, student 
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pressure, or even pressure from colleagues. Our assessment work exposes such patterns of responses in their broader 
institutional context, and we are looking for appropriate ways to respond to such findings. It does seem that this is 
an important, campus-wide role that quantitative assessment can play. 

Assessme it has always had a prominent role in the study of mathematics in colleges and universities. With the 
exception of graduate qualifying examinations, most of this attention has been at the level of individual courses, with 
assessment used to monitor student learning during and at the end of a particular class. The natural focus of 
mathematics faculty members is on ...eir majors and graduate students. We have outlined a locally developed 
procedure that addresses another important but often neglected dimension of assessment in mathematics: student 
retention of mathematical knowledge over the longer term and in relation to the quantitative needs or expiectations 
of other departments. A recent article on the nature of research in collegiate mathematics education ([3]) listed long 
term retention of mathematical knowledge as an important issue deserving study. Although this sort of assessment 
has received little attention in the past, it deserves more prominence because it focuses on the important service role 
played by most mathematics departments. Quantitative assessment also answers the call for more assessment at the 
broader institutional level ([11. [21. and [4]). We believe that our approach helps departments and faculty address 
this important aspect of their educational mission. 
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